>•  tiutm 


AD-A186  037 


nON  PAGE 


a4pp‘  ^  KEAD  INSTRUCTIONS 

_ _ Itt-I  ONK  rOMMLETING  KOHM 

J.  OOVT  ACCCiSION  NO  1-  HeClPICNT*!  CAT  ACOC  NUMaCN 


TiTk  (aA4i 


Necessary  .and  sufficient  conditions  for  the 
convergence  of  integrated  and  mean- integrated 
r-th  order  error  of  histogram  density  estimates 


I*  AUThOM(«j 


X.R.  Chen  and  L.C.  Zhao 


»•  Of  nceoNT  •  vcNiou  covcnco 

g  nv  L 

4o<ihnira.l  -  April  1987 


«.  PCRFORminC  OHC.  RtPORT  numhLH 

87-07 


1.  contract  on  chan r  NUMaep(>t 

F49620-85-C-0008  AIR-P^RCE 


I-  PcnfORMiNO  orCanizatiqn  namc  aho  aoorcss 

Center  for  Multivariate  Analysis 
Fifth.  Floor  Thackeray  Hall 

University  of  Pittsburgh,  Pittsburgh,  PA  15260 


t».  CQi^TRQLLINO  Qf/iCe  namc  AnQ  AQQflItU  ^ 

Alp  Force  Office  of  Scientific  Research 
«  Department  of  the  Air  Force 
Bolling  Air  Force  Base,  DC  20332  O  PTn 


14.  MQNITOAihG  AC£hCY  name  4  AOQRE&&(<<  from  CoA<fOllin|  Off*co> 


S'C^^YVvC’  CvD 


I*.  OlSTHiauTlON  ST  ATCmCN  T  (•/  IA(« 


10.  program  EueMCNT.  PROJECT  TASK 
AReA  *  WORK  unit  NUMBCRS 

U>nosF 

A  A5 


It.  ReRQRT  DATS 

April  1987 


•  J.  number  Of  PACES 

16 


IS.  security  CUASS.  (•<  lAi*  ffpofo 

Unclassi fied 


llo.  DkTCL  ASSiFlCATION/OOWNCRAOlNC 
SChIDuLC 


Approved  for  public  release;  distribution  unlimited. 


17.  OltTRia^TiON  statement  foliA«  «4«r«o<l  M  tflocA  20,  H  4lf/«roni  from  B9p9ft) 


It.  supplcmentart  notes 


DUS 


OCT  0  6 1987 


kCt  WOROS  fCo<«<fny«  #«r«r«o  •l4o  f/ nococtori*  on4  fdofiflli'  4r  4lo<A  num49r> 

consistency,  density  estimation,  histogram 


iO  A0S  I  RAC  T  (CiMiilrtwo  f%  0t9m  II  n«co««o«r  onO  lOooMIr  Ar 

Suppose  that  . are  samples  drawn  from  a  population  with  density 

function  f,  and  f  (xl  =  f  (x;X . X  1  is  an  estimate  of  f(xl.  Denote  by  m^^  = 

n  n  1  n 

f  If  (X)  -  flxlTdx  and  M  =  Eirn  )  the  Integrated  r-th  Order  Error  and  Mean 
•'In 

Integrated  r-th  Order  Error  of  f  for  some  r  >  1  (when  r  =  2.  they  are  the  familiar 


I  Inr  1  e  .•  1  r 


Unclassified  — 


and  widely  studied  ISE  and  MISE).  In  this  paper  the  same  necessary  and  sufficient 
condition  for  lim^  =  0  and  lim^  =  0  as.  is  obtained  when  f  lx)  is  the 
ordinary  histogram  estimator. 


Accesslo'n  For 

~NTIS  GKAJcI 

k 

DTIC  TAB 

o . 

U;if;rinounoed 

□ 

Justlf ioatlon — 

3y__ - - - 

[  Ulstributton/  _ _ 

j  Availability  Codas 
:  ~  (Avail  and/or 
Ol.'it  1  Special 


-t,:*  ■  i 

i  / 


AFOSR-TE.  8  7  -  1  i  2  8 


NECESSARY  AND  SUFFICIENT  CONDITIONS  FOR 
THE  CONVERGENCE  OF  INTEGRATED  AND 
MEAN-INTEGRATED  r-th  ORDER  ERROR  OF 
HISTOGRAM  DENSITY  ESTIf’ATES* 


X.R.  Chen  and  L.C.  Zhao 

Center  for  Multivariate  Analysis 
University  of  Pittsburgh 


Center  for  Multivariate  Analysis 
University  of  Pittsburgh 


NECESSARY  AND  SUFFICIENT  CONDITIONS  FOR 
THE  CONVERGENCE  OF  INTEGRATED  AND 
MEAN-INTEGRATED  r-th  ORDER  ERROR  OF 
HISTOGRAM  DENSITY  ESTIMATES* 


X.R.  Chen  and  L.C.  Zhao 

Center  for  Multivariate  Analysis 
University  of  Pittsburgh 


April  1987 


Technical  Report  No.  87-07 


Center  for  Multivariate  Analysis 
Fifth  Floor  Thackeray  Hall 
University  of  Pittsburgh 
Pittsburgh,  PA  V5260 


Research  sponsored  by  the  Air  Force  Office  of  Scientific  Research  under 
Contract  F49620-85-C-0008.  The  United  States  Government  is  authorized  to 
reproduce  and  distribute  reprints  for  governmental  purposes  notwithstanding 
any  copyright  notation  hereon. 


ABSTRACT 


Suppose  that  X  ,  .  .  ,  ,X  are  samples  drawn  from  a  population  with  density 

1  n 

function  f,  and  f  (x)  =  f  (x;X . X  )  is  an  estimate  of  f(x).  Denote  by  m  = 

n  n  1  n  nr 

r  If  (x)  -  flxlTdx  and  M  =  E(m  )  the  Integrated  r-th  Order  Error  and  Mean 

^  *  n  *  nr  nr 

Integrated  r-th  Order  Error  of  f  for  some  r  >  1  (when  r  =  2,  they  are  the  familiar 
and  widely  studied  ISE  and  MISE).  In  this  paper  the  same  necessary  and  sufficient 
condition  for  lim  M  =0  and  lim  m  =0  a.s.  is  obtained  when  f  (x)  is  the 

nr  n^oy  nr  n 

ordinary  histogram  estimator. 


AMS  1980  Subject  Classification  :  Primary  62G05. 
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Suppose  that  X  ,X  are  iid  samples  drawn  from  a  d-dimension-l 

1  n 

population  with  density  function  f.  Let  f  (x)  =  f  (x;X  ,  ,X  )  be  an  estimator  of 

n  n  1  n 

fix)  The  Integrated  Square  Error  (ISE)  and  Mean  Integrated  Square  Error  (MISEI  of  * 

•  I 

are  defined  by 

ISE  =  /  I  f  ^  (x)  -  f  (x)  I  ^dx. 

WISE  =  Ej  -  f  (x)  l^dx 

respectively.  They  are  important  and  widely  used  criteria  in  evaluating  the 
performance  of  an  estimator  f  Quite  a  lot  of  works  appeared  in  the  statistical 
literature  dealing  with  the  asymptotic  properties  of  them,  for  various  types  of 
estimators,  such  as  kernel  estimator,  orthogonal  senes  estimator,  nearest  neighbor 
estimator  etc.  For  example,  a  much  discussed  problem  is  to  find  the  conditions 
under  which  the  assertions 

limMISE  =  0,  limISE  =  O.a.s. 
n-».oo  n-».ao 

(  i) 

are  true.  Various  conditions  have  been  proposed  m  the  literature  which  are  far 
from  necessary 


In  this  paper  we  obtain  the  necessary  and  sufficient  conditions  of  (1)  for  th-’ 
histogram  estimator  In  fact,  we  have  done  more  For  given  constant  r  >  1,  defmp 
the  integrated  r-th  order  error  m  and  mean  integrated  r-th  order  error  M  by 

nr  nr 


m 

nr 


=  / 


f  (x)  [  ^dx,  h  =  E  (m  ) 
'  nr  nr 


We  obtain  the  necessary  and  sufficient  conditions  of  M  -..0  and  m  =  u. 

nr  rv 

as.  In  the  case  of  r=l,  the  problem  has  been  considered  by  Abou- Jaoude'C  1 J 
I  see  also  [4],  pp  19-231 


A  d-dimensional  histogram  is  defined  as  follows  Choose  a  =  la  a 

n  1  n 


e  R  .  h  =  (h 


,h  )  e  R  with  h  >  0,  i-  1 

dn  m 


of  those  X  ,  X  falling  into  the  set  A  (c 

!  n  ^  n  1 


,  d  Denote  by  k  ttm  nun'bn' 

,  (1) 

,c  I  =  {x  =  (x 
d 


L-  '■> 

ch  <x  <a  +lc 


llh  1=  1 


d} 


(d! 

.X  )  a 


T 


Define 

f  (x)  ■  k/  (nh 


1  n 


h  .  ) ,  when  x  e  A  (c , , 
dn  n  1 


c, ,  ...  ,c  =  0,  +  l ,  +2, 
1  d  ~ 


’V’ 


(2) 


which  is  a  histogram  estimator  of  f(x). 

Theorem.  Suppose  that 

/  f^{x)dx  <  OD,  for  some  r>l 


1  i  m  max  ,h  .  =0 

n-».ool<i<d  in 


I  i  m  nh , 

n-».oo  1  n 


dn 


Then  for  the  histogram  f  defined  by  (2|  we  have 

n 

I  im m  «  0 ,  a .  s  . 
n-*.a3  nr 


I  i  m  M  *  0 , 
n-*.a>  nr 


(3) 


14) 


(5') 


(6) 


(7) 


Conversely,  if  (6)  or  (7)  is  true,  then  (3),  (5)  are  true.  Further,  if  f(x)  does  not  have  a 
form 

'  (x) 


f  (x)  =  2_j  k  (■  ■  \ 

^ — '  h-  M  A  (i  ,  .  ,  I  ) 


Then  (4)  is  also  true.  Here  b 


IS  a  constant.Ali  , 

I  1 

d 


(1) 


(d),, 

,x  ) 


a  +ih  <  X  <a  +(i+1)h,j=1. 
jO  I  jO  lO  )  )0 


,i  )  =  {x  =  (x 
d 


.d}.  and  I  (x) 

B 


denotes  the  indicator  of  B 


w 


Proof.  For  simplicity  of  writing,  we  shall  give  only  the  proof  for  the  case  of 
d=1,  as  the  general  case  involves  no  essential  change.  For  d=1  and  writing  h  <or 

n 

h  .  the  conditions  (4  )  and  (5  )  reduce  to 

n 

1  i  m  h  *0 
n-v®  n 

(1*1 


yy  a  .  >■■  w  .•>  w  .v<v  jvV>'CVKV2r).'0v  Wt J 


1  i  m  nh  =  oo 

n-*.oo  n 


(5) 

In  the  sequel  we  shall  often  write  h  for  h  ,  a  for  a  .  Note  that  we  may  assume 

n  n 

|a|  <  h  without  loss  of  generality.  Write 

=  A  -  =  [a  +  Ah, a  +  {£  +  l)h), 

X  nx 

I  -  (x)  *  I  (x)  , 

p  -  /  f  (x)dx 
^A 

Then  Ef  (x)  =  p./h,  when  x  e  The  symbol  "C"  will  be  used  to  denote  any 

^  A»  Xf 

constant  not  depending  on  n  (but  may  depend  on  r). 

The  proof  will  be  divided  into  four  parts. 

(A).  Sufficiency  of  (3)-(5)  for  (7) 


First  consider  the  case  of  r  >  1.  In  order  to  prove  (7),  we  need  only  to 
show  that 

J  -  Ef^(x)l''dx  =  n''’h  | 

(8) 

/  I  f  (x)  -  Ef ^  (x)  I  ""dx-t-O 

(9) 

as  n  +  00,  (9)  is  easy  to  prove  :  Suppose  first  that  f  is  continuous  on  l-oo.oo),  ttien 
for  any  fixed  constant  A  >  0, 

A 

/  I  f  (x)  -  Ef  ^  (x)  I  ^dx-*.0,as  n-*.oo 

On  the  other  hand,  by  Holder  inequality,  for  any  fixed  constant  B  >  0  we  have 
/  (Ef  (x) )  ^dx:/^  C 

h  " 

«  I  {h'"  ■"  ’  (  /  f  (x)dx)":A.c;[-B,B]''} 

£  I  {/  f^  (x)  dx:  A.C  [-B,B]^} 
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4CMd 


<  f  f  (x)  dx 


Hence 

{  /  |f  (x)  -  Ef  (x)  rdx:A.C[-B.B]^} 

<  2''{  ;  f''{x)dx  +  [  (/  (Ef  (x))"dx:A-C=C-B,B]^} 

_  A  ^  ^ 

[-B.B] 

<  2^  ^  >  /  f^x)dx 

[-b.b]'" 

Summing  up  the  above  arguments  and  noticing  (3),  we  obtain  (9).  For  the  general 

case,  choose  a  function  g  such  that  /  |f(x)  -  g(x)|'^dx  <  some  given  e  >  0,  define 

g  (x)  =  h  ^  r  g(x)dx  for  x  e  A.,  £  =  0,  +1,  ..,  an  argument  similar  to  that 

A^  £ 

leading  to  (10)  gives 

/  (x)  l^dx  <  /  |f  (x)  -  g  (x)  l^dx  <  e 

From  this  and  the  fact  that  (9)  is  true  when  f  is  continuous,  (9)  follows  easily. 

For  a  proof  of  (81,  pot  p^^.  =  IV*!'"  Pfl''  ''r.  ' 

1=  1 

J]  |S  p  f  If  1  <  r  <  2,  then  from  the  inequality  |  1  +  <  1  +  ra  +  Clal"^  (a: 

rea£),  we  have 

Therefore, 

MS  .r  i  E|S„.,  ,1''  +  CE|Y„J''  <  C[  E|Y,  <  2E"P« 


which  in  turn  implies 


n  '"h  *"  VsuP  ET.  <  n  '"h  2Cnp„  =  2C  (nh)  0 

ISlin  jr  -  k  S, 


Suppose  that  (11)  is  true  for  r  e  (1,  m],  we  proceed  to  show  that  it  is  true  for  r 
E  (m,m+1].  Since  when  r  >  2  we  have  the  inequality 

|1  +  a|^  <  1  +  ra  +  Ca^(l  +  |aj''  (a:  real) 

It  follows  that 


'm  «Ti  ,  p_ 


. 

v;.r.v  V,. . 
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■S  ,  „Y 


IS  r  <  Is  I  +  r|S  I  S  Y 
I  n£l  -  '  n-l.ill  I  n-l.x'  n-l.£  n£ 

+  C|S  ,  „r'^Y  +  ClY  l'" 

’  n-1  ,  A  '  n£,  '  njl ' 


which  in  turn  implies 


n-l.£ 


r  +  CElS  ,  J^'^EY  +  CElY  l’’ 


Noticing  that  p^,  E|Y  ±_  p^^,  we  have 


E|S  1^  <  E|S  r  +  Cp  (E|S  1'"^+ 
'  f\V  -  '  R-l,a'  '  Pi-1. a' 


1)  ,  n  =  n, n- 1  , 


Therefore 


E|S  .r  <  Ep.  I  E|E,.r'^  +  PP.E 


-r  -r+1 


Since  f  is  a  probability  density  and  /f^xldx  <  00,  r  >  2,  we  have  Jf  \x)dx  <  <». 

Hence  when  r  >  2 

-  ,  -r+2  r- 1  i-.r"  1  /  \  j  .. 

y  h  1  (x)dx  =  C  <  00 


By  induction  hypothesis 
-r+l  -r+2 

n  h  .max  ET.  ^0,  as  n  0° 

1<j  <n  J  ,  r-  1 


from  (14)-(16),  we  have 
-r  -r+1 

n  h  .max  ET . 

I<j<n  jr 

r  /  u^  /r  r-l  1/(r-l)  ,  Vcic  ,  r-K  (r-2)  /  (r- 1 ) 

iC(nh)  > 

+  C  (nh) 


r  t  ^  /U  ^  r  r-2  (r-2)  /  (r-l) . 

<  C  (nh)  (h  )  (n  h  ) 

-r+1  -r+2  .  (r-2)/(r-l)  ,  .a -r  +  1 

(n  h  .max  ET .  )  +  C  (nh) 

l<j<n  J,r-1 

_  ,  .  .  - 1  ,  -r+1  -r+2  ,(r-2)/(r-l) 

=  C  (nh)  (n  h  .max  ET .  )  +  C (nh) 

l<j<n  j.r-1 


as  n  00. 


This  shows  that  (11)  is  true  for  r  g  im,m+ 1]  .concluding  the  proof  of  (11) 
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hence  (7),  for  r>l  When  r  =  1  (7l  is  a  consequence  of  (6l  for  r  =  1,  which  we 

are  now  going  to  prove 


(B)  Sufficiency  of  l3t-(5)  for  (6) 


Again  first  consider  the  case  of  r>  1  Define  T  as  before,  then 

nr 

I  =/  jf  (x)-f  (x)  l^dx  =  n  '"h 
nS,  '  n  '  nr 

Hence  we  need  only  to  show  that 

1  i  m  I  =  0 ,  a  .  s  . 
n-»co  nr 


Define 


''jti  ■  hjti 

and  pr  )ceed  to  show  that  for  any  given  e  >  0 


-r.  -r+1 


=  P  (n  h  max  I  V  V  L...T\.  ,  >  e)  < 

K  •J®-'  J"'  ~ 


Cn  ,  i  = 


In  the  following  we  shall  write  -  for  r„,Ti„  Since  {  V 

jx  jx  )Xi  jXi 

2,  3,  .  .  }  is  a  martingale  sequence,  we  have 


k  = 


-U  -Ur  -i«r+i4__  , 
J  <  e  n  h  CE  ( 


,1^  5 


From  an  inequality  of  Rosenthal  (see  [5],  p23). 


iC(Ea  nil  I 


Here  F  is  the  O'field  generated  by  X 
inequality  we  have 


,X  Since  I  U  0  I  <  3  bv 

I  ^  IX 


Jensen  s 


J  ■ 


It  can  be  shovvn  easily  by  using  1 1 3'  that 


-  1  .  i 

Noticing  that  I  h  <  dx  <  od  and  nh  _►  co,  we  have 


"  ^  ^  H  ^  0  )  0  ^ 

=  2  E  J  t  j  t 


^  2—1-  2r 

<  Ln (  ;  p  *n  p  I 

E  £  E 


■>  "1  r  - 

<  C  ^  '  r,  h  / 


*  L  ^  ■'  ^  ^  f.  ^  ,  n  ^  ■  ,  I  ^  ) 

=  2  j.  X-  J  ».  Jf"  j  -  I  .  E  j  -  1  .m  '  J  -  I 


2.  2 


5  Cn  E ("  _^’)'+Cn  E 


i  Pel^:-i  ol) 


j-1  ,  E' 


2,  2 


<  CnV  E("  P.Ti,  ") 
"  ,  =  2  2  2'j-I.E 


( 


The  last  step  is  valid  m  view  of  Jensens  inequality  and  -  1  Therefore,  fron 

£  ^ 


Eti  .  , 

2 

10^- 

4 

J- 

1  t  J 

1 

-  1  ,m 

4 

J  *  1  .  E  j 

7r-2 

and 

^"j-1 

l.£  ^ 

4  1/2 

/  I 


We  obtain 


'  j-1 


^  r  r  2^  ^  ^  V  r  '♦r  ’/2 

<  Cn  j;  y  p^  Eti,_,  ,  +  Cnl  I  PoP„(Eti._,  „  Eti._,  ^  ) 

1  =  2  * 


J  =  2 
n 


j  =  2 
2 


A  'j-1.  A 

>A  '• 


j  =  2  A 


,=2  AM,  ^  -j-’*'" 

+  Cn  y  (J  p  (l+{np  )  ’)  ^ 

j=2  A 


r- 1  p  r.  -r+l ,  2 
A 


<  Cn‘(l  +  (nh)^''"^J  'I  P"'  ^  '  ’ 

<  Cn^{nh)^^"^ 


From  (19)-(21)  and  (23),  (18)  follows. 


Now  to  prove  that  for  r  >  1  and  given  e  >  0,  we  have 


_,-r^-r+I  _  .  ,,2 

P  (n  h  ,niax  T,  >  e)  <  C/n 
l<k<n  kr  ~  ~ 


First  suppose  that  1  <  r  <  2.  We  have 

|S  <  IS  ,  j’’  +  C|Y  +  r|S  ,  j'’‘’sign(S  ,  JY  „ 

'  nA'  ”  '  n-l,A*  '  nA'  '  n-l,A'  n-l,A  nA 


Hence 


IS.J'  <  c  E  Iy^oT  +  r  I  Y^^n,. 


J=1 


jA 


j=2 


JA  'j-1  ,A1 


and 


-r  -r+1 

n  h  .fnpx  T, 
l<k<n  kr 


r  -r -f+i 
<  Cn  h 


I,  I 

k 

+  n  '  h  '  '  '  r,max  |y  yY.„Ti. 

2<Kln'^^2  A  jA  'j-1,Al  ' 


■r.  -r+1 


Since 


-r  -r+1 
n  h 


■■  2  (nh)  ■■■■"’.  0 


From  (18)  and  (25),  (24)  follows. 


(23) 


(21*) 


(2  3) 


Suppose  that  (24)  is  true  for  r  <  m,  and  proceed  to  show  that  it  is  true  for 
r  E  (m,  m  +  1].  Since  r  >  2,  we  have 


n-l.£i 


n- 1 , £  nA 


'  njl '  ~  '  n-  1  ,  £  '  ' ' 

+  CIS  ,  .r'^Y  Z+ClY  „  I 
'  n- 1 , £ '  n£  '  n£ 


Hence 


-r  -r+1  -r  -r+1 

n  h  ^mgx  T.  <  Cn  h 


t<n  kr  ~ 


-r  -r+1 
+  rn  h 


i,  5  'V 

k 

k 

+  C  (nh)  '  '.max  Y  P„|S.  ,  „  I =  7  H. 

2<j<n^  £'  j-1  ,£'  I 


^  -r  -r+1 
+  Cn  h 


-r+1 


(26) 


,r-2 


where  ti  ,  =  |S  Observe  that  H  /r  is  the  expression  in  (18),  nhH  /C  is 

j-1.£  '  1-1, £'  2  ^  3 

also  of  the  form  in  (18)  with  r  replaced  by  r  -  1  Further,  since  r  >  2  and 


,r-l 


/f  (x)dx<oD,  we  have  /f  (x)dx<  <=.  Therefore,  the  inequality  (18)  can  be  applied  to 


both  H  and  nhH  .  Further,  nh  <=,  so  we  obtain 
2  3 


P  (H2+H^>e)  <  C/n' 


(27) 


for  arbitrarily  given  e  >  0  Also, 


-r  -r+1 
n  h 


I  I'jul"  i 


-r+1 


(281 


1 


Since  /f  (x)dx  <  oo,  by  induction  hypothesis,  we  have 


.  /  -r+I.  -r+2  _  \  /  2 

P  (n  h  max  T,  ,  >  e)  <  C/n 
1<k<n  k,r-l  -  - 


(29) 


..  —  r-1  r-2 

By  Holder  inequality  and  the  fact  ^  Pp  <  Ch 

£  ^ 


■r+l 


r-2 


I  f  -r+1  -r+2  .(r-2)/(r-l) 

<C  (nh)  (n  h  .max  T.  J 
-  1<k<n  k.r-1 


(30) 


(29),  (30)  together  give 


A  M  JL  i 


P(H^  >  e)  <  C/n 


for  arbitrarily  given  e  >  0.  Summing  up  (26)-(28)  and  (31)  we  see  that  (24)  is  true 
for  r  e  (m,  m  +  1],  concluding  the  proof  of  (24)  hence  (17).  Thus  we  have  proved 


(6)  for  the  case  of  r>1. 


Now  consider  the  case  of  r  =  1.  Since  f  (x)  as  a  function  of  x  in  (-oo,  cn)  is 


a  probability  density,  in  order  to  prove  (6),  it  suffices  to  show  that  for  any  fixed 


positive  integer  N,  it  is  true  that 


limj  |f  (x)-f(x)|dx  =  0,  a.  s. 


n^oo'  •  n 


where  1=  U  A.,H=[h^],  the  integer  part  of  h  \ 
jl  =  -NH  ^ 


To  prove  (32),  it  suffices  to  verify  the  following  two  assertions: 


I  f  (x)  -  Ef^(x)|dx  =  0 


limf  If  (x)  -  Ef  (x)  Idx  *  0,  a.  s. 
n^oo-'  I  n  n  ' 


The  second  assertion  follows  directly  from  Lemma  3  of  Devroye  [3]  if  we  note 
the  fact  that  NH/n  <  N/nh  -►  0.  The  first  assertion  can  be  verified  by  using  a 
continuous  function  g  on  [a-N- 1,a+N+ 1]  such  that  the  integral 


a+N+  1 

/  |f  (x)  -  g  (x)  [dx 
a-N-  1 

does  not  exceed  given  e  >  0.  Trivial  details  are  omitted. 


(C)  Necessity  of  (3),  (4) 


Since 


/  f^"(x)dx  -  n  "h  I  -  P^)r 


<  n  '’h  I  [  P  ''  1  2'’/h'’  '  <  00 


Therefore,  if  (6)  or  (7)  is  true,  then  (3)  is  true 
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Suppose  that  f(x)  does  not  have  a  form 


for  some  a  ,  h  >0  and  {C„}.  We  want  to  prove  that  if  (6)  or  (7)  is  true,  then  h 

0  0  A  ^  r 


0.  Suppose  in  the  contraty  that  h  V  0.  Then  there  exists  subsequence  {h  ,  i  =  1, 


2,  .  .  .  }  such  that  h  -*.h  >0asi-voo.  h  must  be  finite,  otherwise  we  would 

n  0  0 


have  f  (x)  -►  0  uniformly  in  {X  X^,  .  .  .  }  and  and  this  contradicts  obviously 


with  any  one  of  (6)  or  (7).  Since  |  a  )  |  <  h  ,  without  losing  generality  we  may 


assume  that  a  a^,  also  finite.  From  these  facts  it  follows  by  the  law  of  large 


numbers  that  if  we  define 


a  +(£+  1  )h 
,  0  0 


f  (t)dt. 


a  +1l.h 
0  0 


X  e  [a  +S,h  .  a  +(£+1)h  ),  £  =  0,  +  i,  2 

0  0  0  0  . 


Then  we  have 


lim  J  |f  (x)  -  g(x)j^dx  =  0,  a.  s. 


for  any  bounded  interval  I.  Since  at  least  one  of  (6)  and  (7)  is  true,  (36)  implies  that 
f  =  g  almost  everywhere  on  (-oo,  oo)  (Lebesgue  measure).  Hence  f  can  be  expressed 
in  the  form  (35),  contradicting  the  assumption. 


(D),  Necessity  of  (5). 


Suppose  in  the  contratry  that  (5)  is  not  true,  then  there  exists  subsequence 


{h  }  such  that  h  ^.0,  nh  -».u<®  We  restrict  ourselves  to  the  discussion  of 

n  n  in 


the  subsequence.  For  simplicity  of  writing  and  without  losing  generality,  we  may 


assume  that  h  -►  0  and 

n 


1  im  nh  =  u  <  «*> 
n->.oo  n 


Since  h  0,  we  have 


lim  f  If  (x)  -  Ef  (x)  Idx  =  0 
n-*.oo  '  n  ' 


(38) 
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In  fact,  as  pointed  out  earlier,  (33)  is  true  for  any  bounded  interval  I.  Since  f(x)  and 
Ef^(x)  are  probability  density  functions  on  (-00,  od),  this  fact  implies  (38)  Now  it  is 
easily  seen  that  if  at  least  one  of  (6)  and  (7)  is  true,  then 

lim  Ef  If  (x)  -  Ef  (x)  Idx  =  0 
n-^oo  •'  I  n  n  ' 

(39) 

In  fact,  if  (7)  is  true,  then  for  any  bounded  interval  I  we  have  by  Holder  inequality 


E/  ■  f  (x)  -►0,  as  n  -►  00 


Since  f(x),  f  (x|  are  probability  density  functions,  it  is  easily  seen  that 

n 

/  |f^(x)  -  f  (x)  |dx  <  2/  |f  (x)  -  f  (x)  |dx  +  2[1-/  f  {x)dx] 

(41) 

From  (40),  (41),  it  follows  that 

lim  E  f  If  (x)  -  f  (x)  I  dx  =  0 
n-^oo  '  n  ' 

(42) 

Now  (39)  follows  from  (38)  and  (42)  If  (6)  is  true,  then  by  Holder  inequality  we 


/  l^x)  -  f^(x)|dx  =  0,  a.  s, 


for  any  bounded  interval  I.  From  (43)  and  the  dominated  convergence  theorem,  we 
again  have  (40)  and  hence  (39). 


First  we  assume  that  u=0.  Write  A  =  U  [X  -h,  X  +h].  By  the  definition,  f  (x)  =  0 

nil  n 

1=  1 

for  X  e  A  .  So  we  have 

n 

E  f  f  (x)  dx  “  1  . 

A  ri 
A 

n 

Denote  by  X(A  )  the  Lebesgue  measure  of  A  Then 

n  n 

X  (A  )  <  2nh  0,  as  n  00, 
n 

which  implies  that 

lim  E  f  f  (x)  dx  ”  0 . 
n^oo  J 
A 

n 


E/  |f^(><)  -f(x)|dx>E/  |f^(x)  -f(x)|dx 

A 
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>  E  /  f  (x)  dx  -  E  f  f  (x)  dx  -V  1 ,  as  n  -►  oo, 

“  A  "  A 

n  n 

From  this  and  (38),  it  follows  that 

lim  Ef  If  (x)  -  Ef  (x)  Idx  »  1, 
n->.oo  '  n  n  ' 

(kk) 

which  contradicts  (39). 


Now  we  assume  0  <  u  <  <»  By  (39),  there  exist  a  sequence  of  positive 


constants,  say  C  ,  such  that  C  -►  oo.  C  /n  -►  0  and 

n  n  n 

C  E  f  If  (x)  -  Ef  (x)  Idx  -  0 
n  '  n  n  ' 


(i*5) 


Write  k  =  [n/C  ].  Then 

n 

C  E  f  If  (x)  -  Ef  (x)  Idx 
n  '  n  n  ' 

■  E|  I  (I  (X.)  -  p  )| 

>  (kh)"V  E|E{  J  (l^{X.)  -  P^)|x,.  .x^}| 

X  1=  1 

“  l\  l^  ()^(X.)  -  p^)). 

(k6) 


Since  h  0  and  kh  <  nh/C  -*■  0,  by  (44)  we  have 

“  n 

1%  (kh)'’y  E|  I  (i^(x.)  -  p,)|  »  1. 

X  1=  1 

On  the  other  hand,  (45)  and  (46)  together  imply 
-  ] 

(kh)  J  E|  j;  (lj^(XJ  -  P^))|  -  0.  as  n  ^  oo. 

Thus  a  contradiction  is  reached,  and  the  proof  of  the  Theorem  is  completed 


Remark.  The  method  of  proof  in  (D)  can  easily  be  adopted  to  the  case  of  ihe 


kernel  estimate. 
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